{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Function approximation with linear models and neural network\n", "* Are Linear models sufficient for approximating transcedental functions? What about polynomial functions?\n", "* Do neural networks perform better in those cases?\n", "* Does the depth of the neural network matter?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import basic libraries" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Global variables for the program" ] }, { "cell_type": "code", "execution_count": 99, "metadata": { "collapsed": true }, "outputs": [], "source": [ "N_points = 75 # Number of points for constructing function\n", "x_min = 1 # Min of the range of x (feature)\n", "x_max = 25 # Max of the range of x (feature)\n", "noise_mean = 0 # Mean of the Gaussian noise adder\n", "noise_sd = 10 # Std.Dev of the Gaussian noise adder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Generate feature and output vector following a non-linear function\n", "The ground truth or originating function is as follows:\n", "\n", "$$ y=f(x)= (20x+3x^2+0.1x^3)\\sin(x).e^{-0.1x}+\\psi(x) $$\n", "\n", "$$ {OR} $$\n", "\n", "$$ y=f(x)= (20x+3x^2+0.1x^3)+\\psi(x) $$\n", "\n", "$$\\text{ where,}\\ \\psi(x) : {\\displaystyle f(x\\;|\\;\\mu ,\\sigma ^{2})={\\frac {1}{\\sqrt {2\\pi \\sigma ^{2}}}}\\;e^{-{\\frac {(x-\\mu )^{2}}{2\\sigma ^{2}}}}} $$" ] }, { "cell_type": "code", "execution_count": 100, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Definition of the function with exponential and sinusoidal terms\n", "def func_trans(x):\n", " result = (20*x+3*x**2+0.1*x**3)*np.sin(x)*np.exp(-0.1*x)\n", " return (result)" ] }, { "cell_type": "code", "execution_count": 101, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Definition of the function without exponential and sinusoidal terms i.e. just the polynomial\n", "def func_poly(x):\n", " result = 20*x+3*x**2+0.1*x**3\n", " return (result)" ] }, { "cell_type": "code", "execution_count": 102, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Densely spaced points for generating the ideal functional curve\n", "x_smooth = np.array(np.linspace(x_min,x_max,501))\n", "\n", "# Use one of the following\n", "y_smooth = func_trans(x_smooth)\n", "#y_smooth = func_poly(x_smooth)" ] }, { "cell_type": "code", "execution_count": 103, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Linearly spaced sample points\n", "X=np.array(np.linspace(x_min,x_max,N_points))" ] }, { "cell_type": "code", "execution_count": 104, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Added observational/measurement noise\n", "noise_x = np.random.normal(loc=noise_mean,scale=noise_sd,size=N_points)" ] }, { "cell_type": "code", "execution_count": 105, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Observed output after adding the noise\n", "y = func_trans(X)+noise_x" ] }, { "cell_type": "code", "execution_count": 106, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", " | X | \n", "Ideal y | \n", "y | \n", "
---|---|---|---|
0 | \n", "1.000000 | \n", "17.588211 | \n", "17.809904 | \n", "
1 | \n", "1.324324 | \n", "27.166903 | \n", "20.405810 | \n", "
2 | \n", "1.648649 | \n", "35.149333 | \n", "40.925275 | \n", "
3 | \n", "1.972973 | \n", "39.211580 | \n", "49.784422 | \n", "
4 | \n", "2.297297 | \n", "37.421554 | \n", "43.224068 | \n", "